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ABSTRACT 

The purpose of this paper is to describe the experiences of a small. Catholic university as it began implementing the 
various functions available through LiveText. LiveText represents a straight-forward tool for collection, analysis, and 
presentation of candidate artifacts. The LiveText platform was expected to allow to do long-term program improvement 
through a systematic approach to assessment. It was also anticipated that LiveText, as advertised, would allow to 
demonstrate the successful achievement of teacher education candidate standards for the upcoming accreditation visit 
of the campus. Complying with accreditation requirements through the use of this web-based tool resulted in more 
immediate and positive, and unanticipated effects on program-wide reflection, instruction, and assessment practices. 
As the "product" of data analysis to provide the necessary information for programmatic discussions was awaited the 
more subtle yet immediate impact the "process" of using the new technology had on the day-to-day operations was 
recognized. While as individual professors, faculty had always considered the careful creation of appropriate assignments 
and assessment tools, the department-wide use of standard templates, scoring guides and forms to foster more 
reflective, coherent and consistent assessment practices. 

This presentation will discuss the various ways that the use of LiveText had on all aspects of the teacher education program. 
Faculty collaboration, clarification of program goals, overlapping assessment tools and systematic application of their 
own revised conceptual framework standards resulted in a more coherent and, a more productive learning experience 

technology plays in teacher education. With an ever 
increasing amount of research, various strands of 
technology use in teacher education have developed 
examining electronic portfolios (Evans, S., Daniel, T„ 
Mikovch, A., Metze, L., & Norman, A., 2006; Sunal, C. S., 
McCormick, T„ Sunal, D. W„ & Shwery, C. S., 2005; 
Woodward, H., & Nanlohy, R, 2004), online learning 
communities (Cosgrove, M. S., 2002; Fottland, H., 2002; 
Strudler, N„ & Wetzel, K., 1999), candidates' use of 
technology to impact K-l 2 learning (Blocher, J. M„ Echols, 


for the teacher education candidates of the campus. 

INTRODUCTION 

The purpose of this article is to describe the experiences of 
a small private university's teacher education program with 
a web-based technology tool originally acquired to meet 
accreditation requirements (NCATE Standard 2) (NCATE, 
2002). Complying with this requirement resulted in 
important positive, if unanticipated, effects on program¬ 
wide reflection, instruction, and assessment practices. 
Literature Review 

A plethora of research studies examine the role that 
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J„ de Montes, L. S„ Willis, E„ 8c Tucker, G ., 2003; Fay, R„ 8c Hill, 
M., 2003), and faculty's use of technology to model best 
practices and increase teaching effectiveness 
(Teclehaimanot, B., 8c Lamb, A., 2005; Teclehaimanot, B., 8c 
Teclehaimanot, B., 2005). 

However a very extensive literature search found a dearth 
of research on the use of technology for accreditation 
purposes, the focus of this study. The authors consider this 
article as a first step to address this important aspect of 
technology use. 

Initial Expectations of Technology 

In 2003, the Department of Education at The Catholic 
University of America (CUA) invested in a web-based 
environment called College LiveText edu solutions™. 
LiveText is a suite of web-based tools for (1) creating 
assignments and scoring guides to assess candidate 
performance, (2) managing data collection and analysis 
on individual, course, program and unit levels, and (3) 
developing an online exhibit room for accreditation visits. 
From lesson plans, portfolios, and papers for scoring 
guides, surveys, and accreditation data reporting, this 
web-based environment offers a wide range of functions 
and capabilities to meet the technological expectations 
set forth by NCATE. 

The original motivation for the acquisition of this technology 
tool was to meet NCATE accreditation requirements for 
documenting systematic assessment practices (Standard 
2) and collecting candidate data concerning knowledge, 
skills, and dispositions (Standard 1) as part of the larger unit 
assessment (NCATE, 2002). Faculty created templates with 
directions describing the preferred structure for each 
assignment as well as scoring guides for assessment 
purposes. The candidates then submitted their 
assignments through the web-based system, and 
qualitative and quantitative feedback from the professors 
was delivered and archived on each artifact. The 


electronic versions of the assignments and candidate 
scores were available not only to the course professors and 
specific candidates but also to program administrators for 
the purposes of data analysis. 

From the stored data it became possible to aggregate 
and to disaggregate scores in many ways at a click of a 
button. Program administrators were able to sort on several 
levels including individual candidates, courses, yearly 
cohorts, majors, by instructors or semesters, the unit as a 
whole or other demographic groups. This data-collection 
allowed for tracking of performance even if various 
sessions of the same course were taught by different 
professors. At the unit level, once the database was 
established, program reports could be easily generated 
using the above mentioned criteria. A particularly desirable 
feature of this web-based environment was that relevant 
artifacts were automatically linked to the tables generated. 


Assessment Summary for Observation Assessment 

Description: This report is created based on all Teacher Educatibn candidates' 
performance on the observation assignment in multiple courses. 

Milestone: All Scoring: All 



Exceeds 

Meets 

Acquiring 

Mean 



ExpectationExpectation 

Skill 

Stdev 


(3 pts) 

(2 pts) 

(1 pts) 



Description 

130 

94 

7 

2.53 

0.56 

Interpretation 
of observed 

92 

119 

19 

2.32 

0.62 

behavior 






Reflection 

and 

evaluation 

87 

119 

12 

2.34 

0.58 

Application 

91 

111 

12 

2.37 

0.59 

Attention to 

140 

75 

10 

2.58 

0.58 

mechanics 



Z_L 




(NAEYC-INI-la) 

Interpretation of 
observed behavior 
(NAEYC-INI-1) 
Reflection and 
evaluation 
(NAEYC-INI-5dl 
Application 
(NAEYC-ll^5d) 
Attenti' 
mecj^nics 
Exceeds 
I Expectation 


130(56%) 94(40%) 7(3%) 


92(40%) 119(51%) 19(8%) 


87(39%) 119(54%) 12(5%) 


91(42%) | 111 (51%) 12(5%) 


140(62%) 75(33%) 10(4%) 


Meets 

Expectation I 


Acquiring 
Sid 


Assessment Summary Rubric: Observation Assessment 
Attention to mechanics Acquiring Skill 
Science Shadow Lesson 

Author: Anna Anderson Type: project 

Assessor: baker Assessed: 2006-01-09 07:10:11 Status: Submitted 

View Assessment | Review 


Fig. 1. Sample data report and field expansion 
to show specific candidate artifacts. 
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This meant that a click on a particular field within a table 
would pull up the related artifacts for that particular field 
(see Figure 1). 

In this electronic environment various reports can use the 
same artifact without requiring multiple copies of the same 
document, and program coordinators no longer have to 
rely on the record keeping of part-time faculty for long-term 
data collection. 

Another advantage of this technology is that classroom 
professors can find samples of outstanding work even if 
they have not taught the course before. Samples can be 
selected based on performance level in a particular area. 
For example, it is simple to find a strong literature review 
even if it is embedded in an average action research 
paper or a detailed description in an average observation 
paper. 

Actual Impact of Technology 

This web-based environment continues to serve as an 
infrastructure for data-based decision making for program 
improvement as had been originally anticipated. What 
was not expected was how the process of designing and 
implementing the various templates, scoring guides, and 
forms improved the program coherence before any data 
were collected. The use of the technology resulted in a 
dramatic improvement in the department's focus on its 
own reflective practice as a unit. The department has 
found that the technology has driven the faculty to face 
questions of on-going concern that have been too easily 
avoided when working in isolation on a specific course 
rather than program with wide outcomes. These questions 
addressed issues such as individual professor versus 
program expectations, individual candidate versus cohort 
versus program performance, anecdotal versus data- 
based decision making, and the systematic application of 
the program's stated conceptual framework. 
Communication 


Prompted by the new technology tool, issues related to 
departmental policies and goals have been debated in 
the liveliest discussions that the faculty could remember for 
more than ten years. While faculty are not expected or 
required to use LiveText for all course assignments, 
templates and scoring guides have been developed for all 
key assessments on the program and unit levels. This 
process has required faculty to compare course and 
program goals to see how their individual courses 
contribute to the overarching structure and to establish 
common expectations for success. While each faculty 
member is encouraged to use his or her own talents and 
interests to present the material in the most effective 
manner, it is essential that common assignments 
(observation papers, lesson plans, etc.) have clear and 
consistent expectations across courses. This is an 
important consideration for courses taught by part-time 
adjunct faculty. As a result there has been improved 
communication between the full and part-time professors. 
Even those professors who teach a single class are trained 
not only on the use of the technology tool but also on the 
department's conceptual framework and program 
specific goals, and areas that had been identified by the 
department as needing attention. 

Collaborative Practice 

A significant increase in meaningful collaborative practice 
has been most clearly seen in the discussions among the 
core teacher education faculty concerning the 
expectations across courses and the purpose of each 
assignment in light of larger programmatic goals. These 
discussions have resulted in significant changes in 
assignments and the explicit role of reflective practice in 
both course content and faculty conversations. One 
example of this type of shared assignment between 
courses is being piloted in the junior level method classes. A 
new form of thematic unit, that assigned in multiple 
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courses, requires explicit use of the conceptual framework, 
cross-disciplinary planning and assessment, and 
references to current practitioner research literature, in the 
new web-based data system, program coordinators will 
have immediate access to data comparing the different 
professors' use of the scoring guide to foster continued 
discussion of expectations, grading, and areas for revision. 
In another type of collaboration, the team of professors 
who have at one time or another taught a specific 
graduate level course entitled Psychology of Learning for 
Diverse Populations, worked together to redesign the key 
assessment associated with the course. This particular 
assignment is an observation paper in which candidates 
are expected to explicitly tie the theories discussed in the 
course to P-12 student behavior and/or performance to 
make recommendations for future intervention. The new 
project requires candidates to use the departmental 
conceptual framework to focus and guide their reflection, 
prompting them to consider multiple perspectives and 
moral implications of their instructional choices. This 
collaboratively designed key assessment reflects the 
shared expectations of all relevant faculty and allows for 
comparisons through systematically collected data. 
Conceptual Framework 

CUA's Conceptual Framework, which guides CUA 
candidates' reflection and faculty's reflective approach to 
teacher education, places the learner (both the teacher 
candidate and the P-12 student) at its core and 
incorporates the technical aspects of teaching with moral 
considerations to facilitate deliberation of the 
multifaceted nature of every learning environment. It 
addresses three dimensions in education: components of 
reflective decision-making, elements of the learning 
environment, and educational dilemmas (Vaccaro, E. & 
Nagy-Rado, A., 2006). 

The faculty's increased attention to common assignments 


and expectations has allowed the department to address 
the overt application of the conceptual framework in each 
required course and key assessment. Each of the 
sophomore level classes, for instance, presents the 
conceptual framework as a whole and then provides 
scaffolded assignments to encourage the candidates to 
explore specific aspects of the framework through 
meaningful field experiences. A course on the role of 
education in society emphasizes the types of dilemmas 
that face all educators through observations of local 
schools, while the course on educational psychology 
requires candidates to consider the multiple elements of 
any learning environment through their own tutoring 
experiences. Each key assignment explicitly requires 
candidates to use the conceptual framework in their 
written reflection, and the scoring guides include specific 
grading criteria to indicate candidates' developing 
reflective skills. 

Standards 

One additional advantage of the web-based environment 
is the ability of both professors and candidates to easily 
access any set of standards including those of Specialized 
Professional Associations (SPA) and those for P-12 students in 
any state. The technology allows candidates to effortlessly 
include the relevant P-12 standards in their own lesson or 
unit plans, improving their own understanding of how 
standards can be used to support instructional planning. 
Faculty members have created a set of standards 
specifically addressing CUA's conceptual framework, 
which has also been uploaded into the technology tool. All 
documents in this web-based environment including 
templates for directions, scoring guides for assessment, 
syllabi, evaluation forms, and surveys have been 
meticulously aligned by faculty with relevant SPA and CUA 
conceptual framework standards. 

Having ready access to these standards has resulted in a 
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shift in the role the standards play in the department. 
Previously, standards were reserved for program level 
analysis. 

In previous accreditation cycles, individual program 
coordinators took the responsibility of documenting how 
course materials, experiences, and assignments 
adequately prepared candidates for the profession. While 
that has not changed, a significant increase has been 
observed in the level of engagement by the rest of the 
faculty. Each faculty member must consider what sets of 
standards apply and how achievement of specific 
standards is being assessed when creating course syllabi 
and requirements. It is now possible for faculty wide 
discussions on the role of standards to take place in a 
manner that was inconceivable three years ago when 
such discussions would be limited to abstract debates at 
faculty meetings. With the introduction of the technology 
tool, inclusion of the standards became an expected 
practice in the creation of any document. This improves 
not only programmatic assessment but also the quality of 
modeling provided for candidates who expect to 
document achievement and planning using P-12 
standards. 

On-going development/Future plans 

The program coordinators have found the new tool as 
invaluable in preparation for CUA's upcoming 
accreditation visit and have been even more pleased at 
the resulting improvement seen in the quality of the 
teacher education programs. It is important, however, to 
note the challenges the technology has presented to 
many faculty members. Service providers at LiveText have 
been very responsive to the needs of the department and 
have made important changes to the user interface and 
functionality of the technology as needs evolved and 
accreditation requirements changed. Even though these 
changes have been initiated by the LiveText user 


community and, therefore, address specific weaknesses 
identified by the users, there has been a corresponding 
challenge of training faculty to keep up with the updates 
when they use the system fairly sporadically. It is hoped that 
the demands on the faculty will be reduced, as available 
departmental resources and the number of local "experts" 
who can help the more occasional user is growing each 
semester. 

Those faculty members who have participated in cross¬ 
course discussions of goals and expectations have been 
able to see the advantage of using a more systematic, 
electronic, program-wide data collection system. 
Professors who have had a more peripheral role in the 
teacher education program tend to view this technology 
as an obstacle or an added responsibility rather than as a 
tool for improvement. The initial expectations for the system 
were almost completely aimed at program assessment 
and accreditation. Program administrators hope that with 
continued exposure and support, these faculty members 
will come to value the system as a key element in their own 
reflective practice. As they become aware of an authentic 
purpose for the data collection beyond that which can be 
viewed as an arbitrary, or even meaningless, requirement 
imposed by an external accreditation agency, they 
become more active participants in this growing 
conversation. 

Conclusion 

The initial purpose for instituting the data collection system 
was to satisfy the NCATE requirement for documenting and 
assessing unit performance. By using a web-based data 
collection tool, CUA has been able to organize candidate 
artifacts and analyze performance far more simply and 
completely than was thought possible. The program 
coordinators had assumed that the data analysis would 
allow to make long-term improvements to the teacher 
education programs based on the data collected. What 
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had not been anticipated was the much more immediate 
effect that the use of the technology had on instruction 
and assessment. 

Before the advent of the new technology tool, faculty had 
discussed the idea of each course as part of a larger whole 
as the need for more intradepartmental conversations on 
philosophy and goals arose. As it is often the case, 
however, these conversations were not embedded in 
concrete terms with productive outcomes but only as 
abstract concepts. The use of the technology has fostered 
an environment in which meaningful conversations on the 
role of the conceptual framework, instruction, and 
assessment have been translated into concrete plans 
explicitly addressing programmatic improvement, 
implemented at all levels of the teacher education 
program. For the first time in years, the teacher education 
program is using the reflective practice model that faculty 
have been presenting to CUA candidates. Data based on 
candidate artifacts is fueling discussion on the needs and 
interpretation of various stakeholders in light of larger 
departmental goals and expectations. The teacher 
education program's lofty goal is to see this type of 
technology infrastructure that reflects in all other 
departmental programs, from the para-education 
program to the Ph. D. level preparation. In a small 
department, where there is significant overlap between the 
teaching responsibilities in various degree programs, the 
teacher education program administrators hope that as 
other faculty members would come to see the technology 
as a meaningful component of teacher education 
program improvement, and they hope to expand the use 
of this type of system to non-teacher education programs 
as well. 

It is evident that the new and improved technological 
capabilities have allowed faculty to make informed 
decisions about the CUA teacher education programs, 


but, perhaps even more importantly, these technological 
innovations have also made it possible to completely 
implement the educational philosophy of reflective 
practice that is taught in courses but had been 
incompletely modeled to CUA candidates in the past. 
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